
CHAPTER 20. DEEP GENERATIVE MODELS
(2014) identified non-convergence as an issue that may cause GANs to underfit.
In general, simultaneous gradient descent on two players’ costs is not guaranteed
to reach an equilibrium. Consider for example the value function
v
(
a, b
) =
ab
,
where one player controls
a
and incurs cost
ab
, while the other player controls
b
and receives a cost
−ab
. If we model each player as making infinitesimally small
gradient steps, each player reducing their own cost at the expense of the other
player, then
a
and
b
go into a stable, circular orbit, rather than arriving at the
equilibrium point at the origin. Note that the equilibria for a minimax game are
not local minima of
v
. Instead, they are points that are simultaneously minima
for both players’ costs. This means that they are saddle points of
v
that are local
minima with respect to the first player’s parameters and local maxima with respect
to the second player’s parameters. It is possible for the two players to take turns
increasing then decreasing
v
forever, rather than landing exactly on the saddle
point where neither player is capable of reducing its cost. It is not known to what
extent this non-convergence problem affects GANs.
Goodfellow (2014) identified an alternative formulation of the payoffs, in which
the game is no longer zero-sum, that has the same expected gradient as maximum
likelihood learning whenever the discriminator is optimal. Because maximum
likelihood training converges, this reformulation of the GAN game should also
converge, given enough samples. Unfortunately, this alternative formulation does
not seem to improve convergence in practice, possibly due to suboptimality of the
discriminator, or possibly due to high variance around the expected gradient.
In realistic experiments, the best-performing formulation of the GAN game
is a different formulation that is neither zero-sum nor equivalent to maximum
likelihood, introduced by Goodfellow et al. (2014c) with a heuristic motivation. In
this best-performing formulation, the generator aims to increase the log probability
that the discriminator makes a mistake, rather than aiming to decrease the log
probability that the discriminator makes the correct prediction. This reformulation
is motivated solely by the observation that it causes the derivative of the generator’s
cost function with respect to the discriminator’s logits to remain large even in the
situation where the discriminator confidently rejects all generator samples.
Stabilization of GAN learning remains an open problem. Fortunately, GAN
learning performs well when the model architecture and hyperparameters are care-
fully selected. Radford et al. (2015) crafted a deep convolutional GAN (DCGAN)
that performs very well for image synthesis tasks, and showed that its latent repre-
sentation space captures important factors of variation, as shown in figure 15.9.
See figure 20.7 for examples of images generated by a DCGAN generator.
The GAN learning problem can also be simplified by breaking the generation
701